Papers by Nicola De Cao
Learning to Plan and Generate Text with Citations (2024.acl-long)
Copied to clipboard
Constanza Fierro, Reinald Kim Amplayo, Fantine Huot, Nicola De Cao, Joshua Maynez, Shashi Narayan, Mirella Lapata
| Challenge: | Large language models (LLMs) are increasingly useful in information-seeking scenarios, ranging from answering simple questions to generating responses to search-like queries. |
| Approach: | They propose to use plan-based models to improve faithfulness, grounding, and controllability of generated content and its organization. |
| Outcome: | The proposed models improve faithfulness, grounding, and controllability of generated content and its organization. |
GenIE: Generative Information Extraction (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to open information extraction only work with unrealistically small numbers of entities and relations. |
| Approach: | They propose to use a transformer encoder-decoder model to extract triplets from unstructured text . they use 'generative information extraction' to generate triplet representations of information . |
| Outcome: | The proposed model is state-of-the-art on closed information extraction and generalizes from fewer training data points than baselines. |
Multilingual Autoregressive Entity Linking (2022.tacl-1)
Copied to clipboard
Nicola De Cao, Ledell Wu, Kashyap Popat, Mikel Artetxe, Naman Goyal, Mikhail Plekhanov, Luke Zettlemoyer, Nicola Cancedda, Sebastian Riedel, Fabio Petroni
| Challenge: | mGENRE is a sequence-to-sequence system for multilingual entity linking . mGenRE is used to solve language-specific mentions to a multilingual Knowledge Base . |
| Approach: | They propose a sequence-to-sequence system for multilingual entity linking . they match language-specific mentions against a multilingual Knowledge Base (KB) mGENRE is a sequential system that predicts the name of the target entity token-by-token . |
| Outcome: | The proposed system improves on three popular MEL benchmarks and shows improvements in accuracy. |
KILT: a Benchmark for Knowledge Intensive Language Tasks (2021.naacl-main)
Copied to clipboard
Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, Vassilis Plachouras, Tim Rocktäschel, Sebastian Riedel
| Challenge: | Existing models for knowledge-intensive language tasks require access to large, external knowledge sources. |
| Approach: | They propose a benchmark for knowledge-intensive language tasks (KILT) they test a shared dense vector index coupled with a seq2seq model to generate disambiguated text. |
| Outcome: | The proposed model outperforms tailor-made approaches on fact checking, open-domain question answering and dialog by generating disambiguated text. |
Editing Factual Knowledge in Language Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | KnowledgeEditor can be used to edit factual knowledge stored in Language Models without the need for expensive retraining or fine-tuning. |
| Approach: | They propose a method which edits factual knowledge implicitly stored in Language Models and uses it to fix 'bugs' and 'obvious errors' they train a hyper-network with constrained optimization to modify a fact without affecting the rest of the knowledge; the hyper-netzwork is then used to predict the weight update at test time. |
| Outcome: | The proposed method can be used to edit factual knowledge without retraining or fine-tuning and can fix 'bugs' or unexpected predictions without the need for expensive re-training or meta-learning. |
Question Answering by Reasoning Across Documents with Graph Convolutional Networks (N19-1)
Copied to clipboard
| Challenge: | Recent research in reading comprehension has focused on answering questions based on individual documents or even single paragraphs. |
| Approach: | They propose a neural model which integrates and reasons relying on information spread within documents and across multiple documents. |
| Outcome: | The proposed model achieves state-of-the-art on a multi-document question answering dataset, WikiHop. |
Highly Parallel Autoregressive Entity Linking with Discriminative Correction (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to EL have been shown to be effective for both Entity Disambiguation and Entity Linking, but they suffer from high computational cost due to a complex (deep) decoder and the need for training on a large amount of data. |
| Approach: | They propose a method that parallelizes autoregressive linking across all potential mentions and relies on a shallow and efficient decoder. |
| Outcome: | The proposed method outperforms state-of-the-art approaches on the English dataset AIDA-CoNLL and is >70 times faster and more accurate than the previous generative method. |
How do Decisions Emerge across Layers in Neural Models? Interpretation with Differentiable Masking (2020.emnlp-main)
Copied to clipboard
| Challenge: | Attribution methods assess the contribution of inputs to the model prediction. |
| Approach: | They propose a method which removes subsets of inputs and a model which is based on hidden layers to make the decision to include or disregard an input token. |
| Outcome: | The proposed method is efficient because it predicts rather than searches the inputs. |